Apparatus and method for video encoding or decoding

ABSTRACT

Disclosed herein are a QTBT split structure allowing blocks of various shapes capable of more efficiently reflecting various local characteristics of video and a method of efficiently signaling the split structure.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.16/505,080, filed on Jul. 8, 2019, which is a continuation ofInternational Application No. PCT/KR2018/000264, filed on Jan. 5, 2018,which is based upon and claims the benefit of priorities from PatentApplication No. 10-2017-0003156, filed on Jan. 9, 2017 in Korea, andPatent Application No. 10-2018-0001062, filed on Jan. 4, 2018 in Korea,the entire content of which is incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to video encoding or decoding forefficiently encoding video. More particularly, the present inventionrelates to a block splitting method capable of more actively consideringvarious characteristics of videos and a technique for signalingcorresponding split information.

BACKGROUND

The statements in this section merely provide background informationrelated to the present disclosure and may not constitute prior art.

FIG. 1 is a conceptual diagram of an exemplary quadtree block structurefor a coding tree unit (CTU). According to the high efficiency videocoding (HEVC) standard, a CTU is divided into coding units (CUs) using aquadtree structure as a coding tree to reflect various localcharacteristics in a video. As illustrated in FIG. 1, a CU splitstructure resulting in the best coding efficiency is selected whilerecursively dividing the CTU having a maximum size of 64×64 up to aminimum size of 4×4 using a quadtree scheme. Each CU is divided intoprediction units (PUs). After the PUs are determined and the predictionoperation is performed, the CU is divided into a transformation unit(TU) for the residual block. In FIG. 1, rectangular PUs divided by adotted line are illustrated.

Recently, a Quadtree plus Binary tree (QTBT) structure has been newlydiscussed, and attempts have been made to reflect various localcharacteristics of video data while eliminating the existing CU, PU, andTU concepts.

SUMMARY

It is an object of the present invention to provide a QTBT splitstructure allowing blocks of various shapes capable of more efficientlyreflecting various local characteristics of video and a method ofefficiently signaling the split structure.

In accordance with one aspect of the present invention, provided is amethod for decoding video data, including receiving a bitstreamincluding a block of encoded video data and split information related tothe block of the video data;

and determining a quadtree plus binarytree (QTBT) split structure forthe block of the video data using the split information. Herein, theQTBT split structure is configured such that a binary tree is rootedfrom a leaf node of a quadtree, wherein the binary tree is defined bysplit types for splitting a parent node into two child nodes, the splittypes including a triangular type and a rectangular type. The methodfurther includes decoding the block of the encoded video data in theunit of blocks respectively corresponding to leaf nodes of the QTBT. Inthe binary tree, child nodes of a parent node split according to thetriangular type are not further split, while child nodes of a parentnode split according to the rectangular split type are allowed to besplit into two child nodes.

In accordance with another aspect of the present invention, provided isan apparatus for decoding video data, including a memory; and one ormore processors. That is, the one or more processors are configured toperform operations of receiving a bitstream including a block of encodedvideo data and split information related to the block of the video data;determining a quadtree plus binarytree (QTBT) split structure for theblock of the video data using the split information; and decoding theblock of the encoded video data in the unit of blocks respectivelycorresponding to leaf nodes of the QTBT.

In accordance with another aspect of the present invention, provided isa method for decoding video data, including receiving a bitstreamincluding a block of encoded video data and split information related tothe block of the video data; determining a quadtree plus binarytree(QTBT) split structure for the block of the video data using the splitinformation; and decoding the block of the encoded video data in theunit of blocks respectively corresponding to leaf nodes of the QTBT.Herein, the QTBT split structure is configured such that a binary treeis rooted from a leaf node of a quadtree, wherein the leaf node of thequadtree is allowed to be split into two child nodes, the two childnodes being a concave block and a convex block.

In accordance with another aspect of the present invention, provided isa method for decoding video data, including receiving a bitstreamincluding a coding tree unit (CTU) of encoded video data and splitinformation related to the CTU of the video data; determining a quadtreeplus binarytree (QTBT) split structure for the CTU using the splitinformation; determining a prediction partition mode for each leaf nodeof the QTBT using the split information; and decoding the block of theencoded video data according to the prediction partition mode for eachleaf node of the QTBT. In the binary tree of the QTBT, a given block maybe split into two rectangles of the same size. The prediction partitionmode allows the leaf node of the QTBT to be predicted by being splitinto two triangles of the same size.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a conceptual diagram of an exemplary quadtree block structurefor a CTU.

FIG. 2(A) is a diagram illustrating a type of block splitting intosquares and FIG. 2(B) is a diagram illustrating a type of blocksplitting into a combination of squares and triangles.

FIGS. 3A and 3B are diagrams illustrating triangular block-split types.

FIG. 4 is a diagram illustrating block splitting in a concavo-convexshape.

FIG. 5 is a block diagram of an exemplary video encoding apparatus inwhich techniques of the present disclosure can be implemented.

FIG. 6 shows an example of a plurality of intra prediction modes.

FIG. 7 is an exemplary diagram of neighboring blocks of a current block.

FIG. 8 is a block diagram of an exemplary video decoding apparatus inwhich techniques of the present disclosure can be implemented.

FIG. 9 is a conceptual diagram of split partitions allowed in BT splitaccording to an embodiment of the present invention.

FIG. 10 is a tree diagram illustrating an example of a method ofallocating bits to four BT split types illustrated in FIG. 9.

FIG. 11A is a conceptual diagram of an exemplary QTBT split structureaccording to an embodiment of the present invention, and FIG. 11B is adiagram illustrating representation of the QTBT split structure of FIG.11A in a tree structure.

FIG. 12 is a tree diagram illustrating an example of a method ofallocating bits to concavo-convex split types illustrated in FIG. 4.

FIG. 13 is a tree diagram illustrating an example of a bit allocationmethod for BT split in which rectangular splits are allowed.

FIG. 14A is a conceptual diagram of an exemplary CU partition and a PUpartition according to an embodiment of the present invention, and FIG.

14B is a tree diagram illustrating the CU partition of FIG. 14A.

FIG. 15 is a flowchart illustrating an exemplary operation of the videoencoding apparatus of encoding a video.

FIG. 16 is a flowchart illustrating an exemplary operation of the videodecoding apparatus of decoding a video.

FIG. 17 is a flowchart showing another exemplary operation of the videodecoding apparatus of decoding video.

DETAILED DESCRIPTION

Hereinafter, some embodiments of the present invention will be describedin detail with reference to the accompanying drawings. It should benoted that, in adding reference numerals to the constituent elements inthe respective drawings, like reference numerals designate likeelements, although the elements are shown in different drawings.Further, in the following description of the present invention, adetailed description of known functions and configurations incorporatedherein will be omitted when it may make the subject matter of thepresent invention rather unclear.

FIG. 2(A) is a diagram illustrating a type of block splitting intosquares and FIG. 2(B) is diagram illustrating a type of block splittinginto a combination of squares and triangles. As shown in FIG. 2(A), theconventional quadtree splitting method can reflect the givencharacteristics of a video (that is, the location, size, and shape of anobject) to a certain extent by dividing a block into squares havingvarious sizes. However, it should be noted that splitting into acombination of squares and triangles as illustrated in FIG. 2(B) maymore efficiently reflect the given characteristics of a video (and thusbe more advantageous for the encoding operation) than the splitting inFIG. 2(A)

FIGS. 3A and 3B are diagrams illustrating triangular block-split types.FIG. 3A illustrates splitting a square block into two triangles of thesame size. FIG. 3B illustrates splitting a rectangular block into twotriangles of the same size. The proposed triangle blocks will be usefulwhen there is a diagonal edge in a rectangle, as in the example of FIG.2(B). That is, triangular areas having similar textures may berespectively set as one block, and thus the encoding efficiency may beenhanced.

FIG. 4 is a diagram illustrating block splitting in a concavo-convexshape. FIG. 4 shows four types of splitting of a square block in aconcavo-convex convex shape. The split types illustrated in FIG. 4 aredistinguished from each other by the relative location of a convex blockwith respect to a concave block. When a video is split into acombination of such concave and convex blocks and rectangles, the givencharacteristics of the video may be more efficiently reflected.

The present disclosure generally relates to a block splitting methodcapable of more actively considering characteristics of a video and atechnique of signaling the corresponding split information. Thetechniques of the present disclosure allow for a more flexible approachcapable of using block partitions of various shapes besides the squareshape or rectangular shape in video coding, thereby providing additionalopportunities to improve coding compression and/or video quality.

FIG. 5 is a block diagram of an exemplary video encoding apparatus inwhich techniques of the present disclosure can be implemented.

The video encoding apparatus includes a block splitter 510, a predictor520, a subtractor 530, a transformer 540, a quantizer 345, an encoder550, an inverse quantizer 560, an inverse transformer 565, an adder 570,a filter unit 580, and a memory 590. Each element of the video encodingapparatus may be implemented as a hardware chip, or may be implementedas software and the microprocessor may be implemented to execute thefunctions of the software corresponding to the respective elements.

The block splitter 510 splits each picture constituting video into aplurality of coding tree units (CTUs), and then recursively splits theCTUs using a tree structure. A leaf node in the tree structure is acoding unit (CU), which is a basic unit of coding. A QuadTree (QT)structure, in which a node (or a parent node) is split into foursub-nodes (or child nodes) of the same size, or a QuadTree plusBinaryTree (QTBT) structure including the QT structure and a BinaryTree(BT) structure, in which a node is split into two sub-nodes, may be usedas the tree structure. That is, QTBT may be used to split the CTU intomultiple CUs.

In the QuadTree plus BinaryTree (QTBT) structure, a CTU can be firstsplit according to the QT structure. The quadtree splitting may berepeated until the size of the splitting block reaches the minimum blocksize MinQTSize of the leaf node allowed in QT. If the leaf node of theQT is not greater than the maximum block size MaxBTSize of the root nodeallowed in the BT, it may be further partitioned in a BT structure. TheBT may have a plurality of split types. For example, in some examples, arectangular split type of splitting a block of the node into tworectangles of the same size may be used. In some other examples, atriangular split type of splitting a block of the node two triangles ofthe same size may be additionally used in BT split. In still otherexamples, a concavo-convex split type of splitting a given block into aconcave block and a convex block may be used. The rectangular split typemay be divided into a horizontal split type and a vertical split typeaccording to the split directions. The triangular split type may bedivided into a down-right split type and an up-right split typeaccording to the split directions. The concavo-convex split type may bedivided into four split types according to the split directions (i.e.,the relative locations of the concave block and the convex block), asshown in FIG. 4.

The split information generated by the block splitter 510 by splittingthe CTU by the QTBT structure is encoded by the encoder 550 andtransmitted to the video decoding apparatus.

Hereinafter, a block corresponding to a CU (i.e., a leaf node of theQTBT) to be encoded or decoded is referred to as a “current block.”

The predictor 520 generates a prediction block by predicting a currentblock. The predictor 520 includes an intra predictor 522 and an interpredictor 524.

The intra predictor 522 predicts pixels in the current block usingpixels (reference samples) located around the current block in a currentpicture including the current block. There are plural intra predictionmodes according to the prediction directions, and the neighboring pixelsto be used and the calculation equation to be used is defineddifferently according to each prediction mode. In particular, the intrapredictor 522 may determine an intra prediction mode to be used forencoding the current block. In some examples, the intra-predictor 522may encode the current block using several intra prediction modes andselect an appropriate intra prediction mode to use from among the testedmodes. For example, the intra-predictor 522 may calculaterate-distortion values using a rate-distortion analysis for severaltested intra prediction modes, and select an intra prediction modehaving the best rate distortion characteristics among the tested modes.

FIG. 6 shows an example of a plurality of intra prediction modes.

As shown in FIG. 6, the plurality of intra prediction modes may includetwo non-directional modes (a planar mode and a DC mode) and 65directional modes.

The intra predictor 522 selects one intra prediction mode from among theplurality of intra prediction modes, and predicts the current blockusing neighboring pixels (which can be referred to as reference pixelsor reference samples) and an equation determined according to theselected intra prediction mode. The information about the selected intraprediction mode is encoded by the encoder 550 and transmitted to thevideo decoding apparatus.

In order to efficiently encode an intra prediction mode informationindicating which of the plurality of intra prediction modes is used asthe intra prediction mode of the current block, the intra predictor 522may determine some of the intra prediction modes that are most likely tobe used as the intra prediction mode of the current block as the mostprobable modes (MPMs). Then, the intra predictor 522 generates modeinformation indicating whether the intra prediction mode of the currentblock is selected from among the MPMs, and transmits the modeinformation to the encoder 550. When the intra prediction mode of thecurrent block is selected from among the MPMs, the intra predictortransmits, to the encoder, first intra prediction information forindicating which mode of the MPMs is selected as the intra predictionmode of the current block. On the other hand, when the intra predictionmode of the current block is not selected from among the MPMs, secondintra identification information for indicating which of the modesexcluding the MPMs is selected as the intra prediction mode of thecurrent block is transmitted to the encoder. Alternatively, the intrapredictor 522 according to an aspect of the present invention may groupMPMs and/or non-MPMs and signal the index of a group to which the intramode for predicting the current block belongs, instead of explicitlysignaling which mode among the MPMs and/or non-MPMs is selected as theintra prediction mode for predicting the current block.

Hereinafter, a method of constructing an MPM list will be described.

While six MPMs are described as constituting the MPM list, the presentinvention is not limited thereto. The number of MPMs included in the MPMlist may be selected within a range of three to ten.

First, the MPM list is configured using intra prediction modes ofneighboring blocks for the current block. In an example, as shown inFIG. 7, the neighboring blocks may include a part or the entirety of aleft block L, an above block A, a bottom left block BL, an above rightblock AR, and an above left block AL of the current block.

The intra prediction modes of these neighboring blocks are included inthe MPM list. Here, the intra prediction modes of the available blocksare included in the MPM list in order of the left block L, the aboveblock A, the bottom left block BL, the above right block AR, and theabove left block AL, and candidates are configured by adding the planarmode and the DC mode to the intra prediction modes of the neighboringblocks. Alternatively, available modes may be added to the MPM list inorder of the left block L, the above block A, the planar mode, the DCmode, the bottom left block BL, the above right block AR, and the aboveleft block AL. Alternatively, available modes may be added to the MPMlist in order of the left block L, the above block A, the planar mode,the bottom left block BL, the above right block AR, the above left blockAL, and the DC mode.

Only different intra prediction modes are included in the MPM list. Thatis, when there are duplicated modes, only one of the duplicated modes isincluded in the MPM list.

When the number of MPMs in the list is less than a predetermined number(e.g., 6), the MPMs may be derived by adding -1 or +1 to the directionalmodes in the list. In addition, when the number of MPMs in the list isless than the predetermined number, modes are added to the MPM list inorder of the vertical mode, the horizontal mode, the diagonal mode, andso on.

The inter predictor 524 searches for a block most similar to the currentblock in a reference picture encoded and decoded earlier than thecurrent picture, and generates a prediction block for the current blockusing the searched block. Then, the inter predictor 524 generates amotion vector corresponding to a displacement between the current blockin the current picture and the prediction block in the referencepicture. Motion information including information about the referencepicture used to predict the current block and information about themotion vector is encoded by the encoder 550 and transmitted to the videodecoding apparatus. The motion information may be generated in an AMVP(advanced motion vector prediction) mode or in a merge mode. In the caseof the AMVP mode, the following information is generated as the motioninformation: a reference picture index, information on the motion vectorpredictor for use in predicting the motion vector of the current block,and a motion vector difference between the motion vector of the currentblock and the motion vector predictor. In the case of the merge mode,information for indicating a neighboring block of the current block isgenerated as the motion information. In the merge mode, the interpredictor 524 constructs a merge list by selecting a predeterminednumber of merge candidate blocks (hereinafter, “merge candidates”) fromthe neighboring blocks for the current block. A merge candidate of whichmotion information is to be used as the motion information about thecurrent block is selected from among the merge candidates included inthe merge list and merge index information for identifying the selectedcandidate is generated. The generated merge index information is encodedby the encoder 550 and transmitted to the decoding apparatus. The interpredictor 524 sets the motion vector and reference picture index of thecurrent block to the motion vector and reference picture index whichhave been set for the selected merge candidate. The inter predictor 524generates, as the prediction block, a block which is indicated by theset motion vector within a reference picture identified by the setreference picture index.

The subtractor 530 subtracts the prediction block generated by the intrapredictor 522 or the inter predictor 524 from the current block togenerate a residual block.

The transformer 540 transforms residual signals in the residual blockhaving pixel values in the spatial domain into transform coefficients inthe frequency domain. The transformer 540 may transform the residualsignals in the residual block by using the size of the current block asa transform unit, or may split the residual block into a plurality ofsmaller subblocks and transform residual signals in transform unitscorresponding to the sizes of the subblocks. There may be variousmethods of splitting the residual block into smaller subblocks. Forexample, the residual block may be split into subblocks of the samepredefined size, or may be split in a manner of a quadtree (QT) whichtakes the residual block as a root node.

The quantizer 545 quantizes the transform coefficients output from thetransformer 540 and outputs the quantized transform coefficients to theencoder 550.

The encoder 550 encodes the quantized transform coefficients using acoding scheme such as CABAC to generate a bitstream. The encoder 550encodes information such as a CTU size, a MinQTSize, a MaxBTSize, aMaxBTDepth, a MinBTSize, QT_split_flag, and BT_split_flag, which areassociated with the block split, such that the video decoding apparatussplits the block in the same manner as in the video encoding apparatus.

The encoder 550 encodes information about a prediction type indicatingwhether the current block is encoded by intra prediction or interprediction, and encodes intra prediction information or inter predictioninformation according to the prediction type.

The inverse quantizer 560 inversely quantizes the quantized transformcoefficients output from the quantizer 545 to generate transformcoefficients. The inverse transformer 565 transforms the transformcoefficients output from the inverse quantizer 560 from the frequencydomain to the spatial domain and reconstructs the residual block.

The adder 570 adds the reconstructed residual block to the predictionblock generated by the predictor 520 to reconstruct the current block.The pixels in the reconstructed current block are used as referencesamples in performing intra prediction of the next block in order.

The filter unit 580 deblock-filters the boundaries between thereconstructed blocks in order to remove blocking artifacts caused byblock-by-block encoding/decoding and stores the blocks in the memory590. When all the blocks in one picture are reconstructed, thereconstructed picture is used as a reference picture for interprediction of a block in a subsequent picture to be encoded.

Hereinafter, a video decoding apparatus will be described.

FIG. 8 is a block diagram of an exemplary video decoding apparatus inwhich techniques of the present disclosure can be implemented.

The video decoding apparatus includes a decoder 810, an inversequantizer 820, an inverse transformer 830, a predictor 840, an adder850, a filter unit 860, and a memory 870. As in the case of the videoencoding apparatus of FIG. 5, each element of the video encodingapparatus may be implemented as a hardware chip, or may be implementedas software and the microprocessor may be implemented to execute thefunctions of the software corresponding to the respective elements.

The decoder 810 decodes a bitstream received from the video encodingapparatus, extracts information related to block splitting to determinea current block to be decoded, and extracts prediction informationnecessary to reconstruct the current block and information about aresidual signal.

The decoder 810 extracts information about the CTU size from a highlevel syntax such as the sequence parameter set (SPS) or the pictureparameter set (PPS), determines the size of the CTU, and splits apicture into CTUs of the determined size. Then, the decoder determinesthe CTU as the uppermost layer, that is, the root node, of a treestructure, and extracts split information about the CTU to split the CTUusing a tree structure (e.g., the QTBT structure).

Upon determining the current block to be decoded through splitting ofthe tree structure, the decoder 810 extracts information about theprediction type indicating whether the current block is intra-predictedor inter-predicted.

When the prediction type information indicates intra prediction, thedecoder 810 extracts a syntax element for the intra predictioninformation (intra prediction mode) about the current block. First, thedecoder 810 extracts mode information (i.e., MPM flag) indicatingwhether the intra prediction mode of the current block is selected fromamong the MPMs. In general, when the mode information indicates that theintra prediction mode of the current block is selected from among theMPMs, the decoder extracts first intra prediction information forindicating which mode of the MPMs is selected as the intra predictionmode of the current block. When the intra prediction mode encodinginformation indicates that the intra prediction mode of the currentblock is not selected from among the MPMs, the decoder extracts secondintra identification information for indicating which of the modesexcluding the MPMs is selected as the intra prediction mode of thecurrent block. Alternatively, the intra predictor 522 according to anaspect of the present invention may group MPMs and/or non-MPMs andextracts intra identification information (e.g., the index of a group)indicating a group to which the intra mode for predicting the currentblock belongs, instead of extracting intra identification informationindicating which mode among the MPMs and/or non-MPMs is selected as theintra prediction mode for predicting the current block. When theprediction type information indicates inter prediction, the decoder 810extracts syntax elements for inter prediction information. The mergeindex information is extracted in the case of the merge mode, while thereference picture index, the information on the motion vector predictor,and the motion vector difference are extracted in the case of the AMVPmode.

The decoder 810 extracts information about the quantized transformcoefficients of the current block as information about the residualsignal.

The inverse quantizer 820 inversely quantizes the quantized transformcoefficients. The inverse transformer 83 inversely transforms theinversely quantized transform coefficients from the frequency domain tothe spatial domain to reconstruct the residual signals, and therebygenerates a residual block for the current block.

The predictor 840 includes an intra predictor 842 and an inter predictor844. The intra predictor 842 is activated when the prediction type ofthe current block is intra prediction, and the inter predictor 844 isactivated when the prediction type of the current block is interprediction.

The intra predictor 842 determines an intra prediction mode of thecurrent block among the plurality of intra prediction modes by using thesyntax element for the intra prediction mode extracted from the decoder810, and predicts the current block using reference samples neighboringto the current block according to the determined intra prediction mode.

To determine the intra prediction mode of the current block, the intrapredictor 842 constructs an MPM list including a predetermined number of

MPMs from the neighboring blocks around the current block. The method ofconstructing the MPM list is the same as that for the intra predictor522 of FIG. 5.

Generally, when the intra prediction mode information (i.e., the MPMflag) indicates that the intra prediction mode of the current block isselected from among the MPMs, the intra predictor 842 selects, as theintra prediction mode of the current block, the MPM indicated by thefirst intra identification information among the MPMs in the MPM list.On the other hand, when the mode information indicates that the intraprediction mode of the current block is not selected from among theMPMs, intra predictor 842 selects the intra prediction mode of thecurrent block among the intra prediction modes other than the MPMs inthe MPM list, using the second intra identification information.

Alternatively, as described above, the intra predictor 522 of the videoencoding apparatus according to an aspect of the present inventiongroups MPMs and/or non-MPMs and signals the index of a group to whichthe intra mode for predicting the current block belongs, instead ofexplicitly signaling which mode among the MPMs and/or non-MPMs isselected as the intra prediction mode for predicting the current block.In this case, the intra predictor 842 of the video decoding apparatusmay determine the final intra mode (i.e., the intra mode for predictingthe current block) by evaluating the intra modes belonging to the group.For example, in some examples, the intra-predictor 842 may generatereconstructed blocks for multiple intra modes belonging to a group, andevaluate the reconstructed blocks to determine a final intra mode.

The inter predictor 844 determines the motion information about thecurrent block using the syntax elements for the inter mode extracted bythe decoder 810, and predicts the current block using the determinedmotion information. In the case of the merge mode, the inter predictor844 constructs a merge list by selecting a predetermined number of mergecandidates from the neighboring blocks for the current block, andselects a merge candidate indicated by the merge index information amongthe candidates included in the merge list. The inter predictor 844 setsthe motion vector and reference picture index of the current block tothe motion vector and reference picture index which have been set forthe selected merge candidate. The inter predictor 844 generates, as theprediction block of the current block, a block which is indicated by theset motion vector within a reference picture identified by the setreference picture index. In the case of the AMVP mode, the interpredictor 844 determines the motion vector predictor using theinformation on the motion vector predictor extracted by the decoder 810,and determines the motion vector of the current block by adding themotion vector predictor and the motion vector difference. The interpredictor 844 generates, as the prediction block of the current block, ablock which is indicated by the determined motion vector of the currentblock within a reference picture identified by the reference pictureindex that the decoder 810 extracts.

The adder 850 adds the residual block output from the inversetransformer and the prediction block output from the inter predictor orintra predictor to reconstruct the current block. The pixels in thereconstructed current block are utilized as reference samples for intraprediction of a block to be decoded later.

The filter unit 860 deblock-filters the boundaries between thereconstructed blocks in order to remove blocking artifacts caused byblock-by-block decoding and stores the deblock-filtered blocks in thememory 870. When all the blocks in one picture are reconstructed, thereconstructed picture is used as a reference picture for interprediction of blocks in a subsequent picture to be decoded.

The techniques of the present disclosure generally relate to a techniquefor signaling split information representing a QTBT block splitstructure that allows not only rectangular blocks but also triangular orconcavo-convex blocks, and a technique for determining blockpartitioning from split information. One aspect of the techniques of thepresent disclosure may be carried out by the encoder 550 of the videoencoding apparatus illustrated in FIG. 5 and/or the decoder 810 of thevideo decoding apparatus illustrated in FIG. 8. In other examples, oneor more other units of the video encoding apparatus and/or videodecoding apparatus may additionally or alternatively be responsible forcarrying out the techniques of the present disclosure. QTBT blockpartitioning may be used to partition a CTU into CUs. That is, the rootnode of the QTBT may be a CTU, and a leaf node of the QTBT may be a CU.

In the QTBT structure, the CTU, which is the root node, may be firstsplit according to the QT structure. The video encoding apparatus uses aQT split flag QT_split_flag to signal whether the root node of the QTBTis split according to the QT structure. As shown in Table 1,QT_split_flag=0 indicates that the root node is not QT-split, andQT_split_flag=1 indicates that the root node is split into four blocksof the same size.

TABLE 1 QT split flag Value No split 0 Split 1

QT split of each node may be recursively repeated. BT split may beperformed on the blocks that are not subjected to QT split (i.e., theleaf nodes of the QT). The BT split proposed in the present disclosuremay have a plurality of split types. For example, in some examples, arectangular split type of splitting a block of a node into tworectangles of the same size may be used. In some other examples, atriangular split type of splitting a block of the node into twotriangles of the same size may be additionally used in BT split. Instill other examples, a concavo-convex split type of splitting a givenblock into a concave block and a convex block may be used.

Hereinafter, some embodiments relating to the split types allowed in BTsplit and a method of signaling the same will be described.

First Embodiment (BT Split-Combination of Rectangular Split andTriangular Split)

FIG. 9 is a conceptual diagram of split partitions allowed in BT splitaccording to an embodiment of the present invention.

In this embodiment, rectangular split and triangular split are used forthe BT split. Rectangular split is splitting a given (square orrectangular) block into two rectangles of the same size, and triangularsplit is splitting a given (square or rectangular) block into twotriangles of the same size. FIGS. 9(a) and 9(b) are examples of twotypes of rectangular split, and FIGS. 9(c) and 9(d) are examples of twotypes of triangular split. The rectangular split is divided intohorizontal split and vertical split according to the split directions,and the triangular split is divided into down-right split and up-rightsplit according to the split directions. Accordingly, in thisembodiment, the BT split is divided into four types according to shapesand directions.

Although FIG. 9 illustrates cases where a square block is divided intotwo rectangles or two triangles, it should be noted that a rectangularblock can be divided into two rectangles or two triangles. Therefore, arectangular block may be further divided into two rectangles or twotriangles. On the other hand, a triangular block is not further split.

FIG. 10 is a tree diagram illustrating an example of a method ofallocating bits to four BT split types illustrated in FIG. 9.

Referring to FIG. 10, one bit is assigned to indicate whether BT splitis selected. When BT split is selected, one bit for indicating the typeof BT split (i.e., whether triangular split is selected or rectangularsplit is selected) and one bit for indicating the split direction areadditionally allocated. Therefore, a BT split type for a given block maybe represented by 3 bits.

Table 2 summarizes the bits assigned to each BT split type. As shown inTable 2, the first bin of the BT split flag indicates whether the blockis subjected to BT split, the second bin indicates the type of BT split(i.e., whether the splitting is rectangular split or triangular split),and the third bin indicates the split direction of the BT split (i.e.,the direction of the rectangular split or the direction of thetriangular split).

TABLE 2 BT split flag Value No split  0 Split rectangle (horizontal) 100Split rectangle (vertical) 101 Split triangle (down-right) 110 Splittriangle (up-right) 111

FIG. 11A is a conceptual diagram of an exemplary QTBT split structureaccording to an embodiment of the present invention, and FIG. 11 B is adiagram illustrating representation of the QTBT split structure of FIG.11A in a tree structure. In FIG. 11B, the black circle indicates a leafnode of a QTBT split structure, and a white circle indicates a nodesubjected to QT split or BT split. In addition, bits marked on each noderefer to the QT split flag and the BT split flag based on Tables 1 and2. Among the bits assigned to each node, the underlined bits are QTsplit flags, and the remaining bits are BT split flags. Note that inFIG. 11B, bit “0” is not assigned to the child nodes of the nodes towhich “110” and “110” in italics are assigned. As described above, atriangular block corresponds to a leaf node that is not further split.Accordingly, when a given node is subjected to triangular split, it isnot necessary to signal that the child node of the given node is notfurther subjected to BT split.

Alternatively, when a block is split into triangular blocks, thetriangular blocks may have child nodes. In this case, the child nodesmay be triangular blocks obtained by bisecting the parent node thereof.The triangular split is independently determined for each of the twotriangular blocks, which are the parent nodes. The corresponding splitflag may be set to “0” if splitting is not performed, and set to “1” ifsplitting is performed.

Second Embodiment (BT split-Combination of Rectangular Split andConcavo-convex Split)

In this embodiment, concavo-convex partitions illustrated in FIG. 4 areused for the BT split. The concavo-convex partitions are classified infour types according to the split directions (that is, according to therelative locations of the convex blocks with respect to the concaveblocks).

FIG. 12 is a tree diagram illustrating an example of a method ofallocating bits to concavo-convex split types illustrated in FIG. 4.Referring to

FIG. 12, one bit is allocated to indicate whether BT split is selected.When BT split is selected, two bits are further allocated to indicatethe type of the selected concavo-convex split. The bits assigned to eachof the split patterns are summarized in Table 3. As shown in Table 3,the first bin of the BT split flag indicates whether the block issubjected to BT split, and the second bin and the third bin indicate thetype of the applied concavo-convex split.

TABLE 3 BT split flag Value No split  0 Split rectangle (horizontal) 100Split down-type 101 Split left-type 110 Split right-type 111

In this embodiment, once a square (or rectangular) block is split in aconcavo-convex shape, the split blocks are not further divided in the BTstructure. Therefore, it is not necessary to signal the BT split flagfor the concavo-convex child nodes of the BT-split nodes.

Third Embodiment (Allowing Triangular Split for PU)

In the technique proposed in this embodiment, the rectangular splitsillustrated in FIGS. 9(a) and 9(b) are used for BT split in the QTBTsplit structure of a CTU for determining a coding unit (CU), and thetriangular splits illustrated in FIGS. 9(c) and 9(d) are used forsplitting of a CU for determining a prediction unit (PU). That is, whilethe first embodiment allows a triangular CU, this embodiment allows atriangular PU. Here, the triangular PU may be described as a partitionmode for predicting the corresponding CU.

FIG. 13 is a tree diagram illustrating an example of a bit allocationmethod for BT split in which rectangular splits are allowed. Referringto FIG. 13, one bit is assigned to indicate whether BT split isselected. When BT split is selected, one bit is additionally assigned toindicate the direction of BT split (i.e., the direction of therectangular split). Therefore, the BT split type for a given block maybe represented by 2 bits. Table 4 summarizes the bits assigned to eachtype of BT split. As shown in Table 4, the first bin of the BT splitflag indicates whether the block is subjected to BT split, and thesecond bin indicates the split direction of the BT split (i.e., thedirection of the rectangular split).

TABLE 4 BT split flag Value No split  0 Split horizontal 10 Splitvertical 11

FIG. 14A is a conceptual diagram of an exemplary CU partition and a PUpartition according to an embodiment of the present invention, and FIG.14B is a tree diagram illustrating the CU partition of FIG. 14A. In FIG.14A, solid lines indicate CU splits in the QTBT split structure, anddotted lines indicate PU splits. In FIG. 14B, the black circle indicatesa leaf node of the QTBT split structure, and the white circle indicatesa node that is subjected to QT split or BT split. In addition, the bitsmarked on each node refer to the QT split flag based on Table 1 and theBT split flags based on Table 4.

Once a CU partition is determined, the video encoding apparatus shouldsignal PU partition information (or mode type) about the correspondingCU. Signaling of the mode type is performed for all CUs corresponding tothe leaf nodes in FIG. 14B. Information about the mode type of one CUmay be indicated as shown in Table 5.

TABLE 5 BT split flag Value Quadrangle  0 Triangle (down-right) 10Triangle (up-right) 11

FIG. 15 is a flowchart illustrating an exemplary operation of the videoencoding apparatus for encoding a video.

In the example of FIG. 15, the video encoding apparatus determines aQTBT spit structure for encoding a block of video data (S1510). The QTBTblock partitioning structure is a structure in which a binary tree isrooted from a leaf node of a quadtree. In some examples, the binary treeis defined by a plurality of split types in which a parent node is splitinto two child nodes. The split types may include a triangular splittype and a rectangular split type. In the binary tree, the nodes thatare generated by the split according to the triangular split type arenot further split. In the binary tree, the nodes that are generated bythe split according to the rectangular split type are allowed to befurther split into two child nodes. In some other examples, a leaf nodeof the quadtree is allowed to be split into two child nodes, and the twochild nodes may be a concave block and a convex block. In this case, thechild nodes may not be further split.

The video encoding apparatus generates an encoded bitstream including ablock of the video data and split information representing thedetermined QTBT split structure, based on the determined QTBT splitstructure (S1520).

FIG. 16 is a flowchart illustrating an exemplary operation of the videodecoding apparatus for decoding a video.

In the example of FIG. 16, the video decoding apparatus receives abitstream including a block of encoded video data and split informationrelated to the block of the video data (S1610).

The video decoding apparatus determines the QTBT split structure for theblock of the video data using the split information (S1620). The QTBTsplit structure is a structure in which a binary tree is rooted from aleaf node of a quadtree. In some examples, the binary tree is defined bya plurality of split types in which a parent node is split into twochild nodes. The split types may include a triangular split type and arectangular split type. In the binary tree, the nodes that are generatedby the split according to the triangular split type are not furthersplit. In the binary tree, the nodes that are generated by the splitaccording to the rectangular split type are allowed to be further splitinto two child nodes. In some other examples, a leaf node of thequadtree is allowed to be split into two child nodes, and the two childnodes may be a concave block and a convex block. In this case, the childnodes may not be further split.

The video decoding apparatus decodes the block of the encoded video datain the unit of blocks respectively corresponding to leaf nodes of theQTBT (S1630).

FIG. 17 is a flowchart showing another exemplary operation of the videodecoding apparatus for decoding a video.

In the example of FIG. 17, the video decoding apparatus receives abitstream including a coding tree unit (CTU) of the encoded video dataand split information related to the CTU of the video data (S1710).

The video decoding apparatus determines the QTBT split structure for theCTU using the split information (S1720). Here, the QTBT split structureis a structure in which a binary tree is rooted from a leaf node of aquadtree. In the binary tree, a given block is split into two rectanglesof the same size. Nodes that are generated by the split according to therectangular split type are allowed to be further split into two childnodes. Thus, the leaf nodes (i.e., CUs) of the QTBT may have a square orrectangular shape.

The video decoding apparatus determines a prediction partition mode foreach leaf node of the QTBT using mode type information (S1730). Theprediction partition mode allows the leaf node of the QTBT to bepredicted by being split into two triangles of the same size. Thus, thePU may have a square, rectangular, or triangular shape.

The video decoding apparatus decodes the block of the encoded video dataaccording to the prediction partition mode for each leaf node of theQTBT (S1740).

Although exemplary embodiments have been described for illustrativepurposes, those skilled in the art will appreciate that and variousmodifications and changes are possible, without departing from the ideaand scope of the embodiments. Exemplary embodiments have been describedfor the sake of brevity and clarity. Accordingly, one of ordinary skillwould understand the scope of the embodiments is not limited by theexplicitly described above embodiments but is inclusive of the claimsand equivalents thereof.

What is claimed is:
 1. A method for decoding video data, comprising:receiving a bitstream including a coding tree unit (CTU) of encodedvideo data and split information related to a tree split structure ofthe CTU, wherein the bitstream includes information on maximum blocksize allowed for a root node of a binary tree; determining a codingblock to be decoded, wherein the coding block is a block which is splitfrom the CTU in the tree split structure using the split information andcorresponds to a leaf node of the tree split structure, the tree splitstructure allowing, depending on the split information, a quadtreehaving multi-depth of consecutive quadtree divisions and a binary treehaving multi-depth of consecutive binary tree divisions, wherein thetree split structure is configured such that the binary tree is rootedfrom a leaf node of the quadtree when a size of a block corresponding tothe leaf node of the quadtree is not greater than the maximum blocksize; after determining the coding block to be decoded by splitting theCTU in the tree split structure, determining a prediction partition modefor the coding block corresponding to the leaf node of the tree splitstructure among a plurality of prediction partition modes using modetype information included in the bitstream, the plurality of predictionpartition modes including at least one prediction partition modeallowing the coding block to be predicted by being split into twoequal-sized triangles; and decoding the coding block corresponding tothe leaf node according to the determined prediction partition mode,wherein, when the coding block is split into the two triangles accordingto the determined prediction partition mode, the decoding of the codingblock comprises: configuring a plurality of merge candidates comprisedof motion vectors of pre-decoded blocks neighboring to the coding block;reconstructing, from the bitstream, two merge indexes which respectivelycorrespond to the two triangles, wherein each of the merge indexes isused for indicating one of the plurality of merge candidates; settingmotion vectors respectively corresponding to the two triangles equal tomerge candidates which are respectively indicated by the two mergeindexes among the plurality of merge candidates; and inter-predictingthe coding block which is split into the two triangles, by using the setmotion vectors, and wherein the at least one prediction partition modeincludes a down-right diagonal split type and an up-right diagonal splittype.
 2. The method of claim 1, wherein the bitstream includesinformation on minimum block size allowed for the leaf node of thequadtree, wherein a block having the minimum block size is not furthersplit in the quadtree.
 3. The method of claim 1, wherein the splitinformation includes information for indicating one among a plurality ofsplit types for binary tree division.
 4. An apparatus of encoding videodata, the apparatus comprising at least one processor configured to:encode restriction information related to a tree split structure, therestriction information including information on maximum block sizeallowed for a root node of a binary tree; split a coding tree unit (CTU)in the tree split structure and encode split information related to thetree split structure of the CTU, the tree split structure allowing aquadtree having multi-depth of consecutive quadtree divisions and abinary tree having multi-depth of consecutive binary tree divisions,wherein the tree split structure is configured such that the binary treeis rooted from a leaf node of the quadtree when a size of a blockcorresponding to the leaf node of the quadtree is not greater than themaximum block size; select, from a plurality of prediction partitionmodes, a prediction partition mode for a coding block corresponding to aleaf node of the tree split structure, and encode mode type informationfor representing the selected prediction partition mode, wherein theplurality of prediction partition modes includes at least one predictionpartition mode allowing the coding block to be predicted by being splitinto two equal-sized triangles; and encode the coding block according tothe selected prediction partition mode, wherein, when the coding blockis split into the two triangles, the processor is configured to encodethe coding block by performing a process comprising: configuring aplurality of merge candidates comprised of motion vectors of pre-decodedblocks neighboring to the coding block; setting motion vectorsrespectively corresponding to the two triangles equal to two mergecandidates included in the plurality of merge candidates; encoding mergeindex information for indicating the two merge candidates which are setas motion vectors respectively corresponding to the two triangles,inter-predicting the coding block which is split into the two triangles,by using the set motion vectors, wherein the at least one predictionpartition mode includes a down-right diagonal split type and an up-rightdiagonal split type.
 5. The apparatus of claim 4, wherein therestriction information includes information on minimum block sizeallowed for the leaf node of the quadtree, wherein a block having theminimum block size is not further split in the quadtree.
 6. Theapparatus of claim 4, wherein the split information includes informationfor indicating one among a plurality of split types for binary treedivision.
 7. A non-transitory computer readable medium storing abitstream generated by a method of encoding video data, the methodcomprising: encoding restriction information related to a tree splitstructure, the restriction information including information on maximumblock size allowed for a root node of a binary tree; splitting a codingtree unit (CTU) in a tree split structure and encoding split informationrelated to the tree split structure of the CTU, the tree split structureallowing a quadtree having multi-depth of consecutive quadtree divisionsand a binary tree having multi-depth of consecutive binary treedivisions, wherein the tree split structure is configured such that thebinary tree is rooted from a leaf node of the quadtree when a size of ablock corresponding to the leaf node of the quadtree is not greater thanthe maximum block size; selecting, from a plurality of predictionpartition modes, a prediction partition mode for a coding blockcorresponding to a leaf node of the tree split structure, and encodingmode type information for representing the selected prediction partitionmode, wherein the plurality of prediction partition modes includes atleast one prediction partition mode allowing the coding block to bepredicted by being split into two equal-sized triangles; and encodingthe coding block according to the selected prediction partition mode,wherein, when the coding block is split into the two triangles, theencoding of the coding block comprises: configuring a plurality of mergecandidates comprised of motion vectors of pre-decoded blocks neighboringto the coding block; setting motion vectors respectively correspondingto the two triangles equal to two merge candidates included in theplurality of merge candidates; encoding merge index information forindicating the two merge candidates which are set as motion vectorsrespectively corresponding to the two triangles, inter-predicting thecoding block which is split into the two triangles, by using the setmotion vectors, wherein the at least one prediction partition modeincludes a down-right diagonal split type and an up-right diagonal splittype.